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ABSTRACT 



j The Use of Judgmental Data in Roll Call Analysis 

Despite the fact that the roll call voting patterns of legislators are being 
I used more frequently in the political arena, the statistical methods employed 
by political scientists for analyzing roll calls (e.g. , Guttman scaling, 
factor analysis) are not being used. The main reason is the failure to incor- 
porate in a systematic manner the subjective estimates of the political 
organizations and individuals who must use the output of such analyses. 

This paper presents two methods, paired comparison and constant sum, which 
use judgmental data in assessing roll calls. Using a set of seven defense 
policy roll calls from the 1976 US Senate, the methods are described and 



compared using two sets of judges . 



THE USE OF JUDGMENTAL DATA IN ROLL CALL ANALYSIS 



Introduction 

The political science discipline is undergoing an obvious change in 
he direction of policy relevance. Conferences and panels increasingly focus 
in the utility of much of what we do for public policymakers. New journals 
ixplicitly focusing on public policy have emerged. Textbooks on American 
[overnment now must have "public policy" somewhere in the title. Government 
unding for research is increasingly contingent on the political scientist demon- 
jtrating the payoff of the proposed research for policymaking. 

One area in political science where this does not appear to be the case 
s legislative roll call analysis. It is the purpose of this article: 1) to first 
show that the use of roll calls in politics is increasing; 2) to demonstrate why 
most of the research in roll call analysis has not been utilized for policymaking; 
3) and put forth several methods for using judgmental data in assessing roll calls 
/i^hich will make this type of analysis more useful in the policy arena. 

Increased Importance of Voting In The Political Arena 

The use of roll calls to describe the performance of Congressmen in 
general and on specific issues is not new. For example, the American Federation 
Df Government Employees has rated Congressmen since the 1940s, and the 
\mericans for Democratic Action since 1948. But it wasn't until the early 1970s 
that the "rating game" swung into high gear and became an important factor in 



election campaigns. The key was the shift from the general orientation approach 
(ADA, Congressional Quarterly's Conservative Coalition, etc.) to one focused 
on specific issues. 

No better example could be given than the impact of the Environmental 

Action's Dirty Dozen campaign. In 1970, 1972, 1974 and 1976 this group used 

roll call analysis to identify and publicize the twelve Congressmen with the 

"worst" anti-environment record. Through 19 74, 31 different Congressmen were 

so identified. Seventeen were defeated in the first year being listed. Five 

were defeated in subsequent elections. Two retired and seven remain in 

Congress.^ In 1976, three of the "dirty dozen" were defeated. 

This use of roll calls to pinpoint Congressmen who are performing 

"poorly" or not in the best interests of certain groups is not limited to the liberal 

causes such as consumerism and environment. For example, in November 1975 

the Atlantic Richfield Company's Civic Action Program (CAP) compiled legislator 

scores on the energy industry's position based on six key energy votes. The 

clearly political intent is obvious since they listed only legislators "who rep- 

2 

resent a substantial number of CAP members . " They singled out Rep. Robert 

W. Edgar (D-PA) for special attention, noting that his 

votes since election can be characterized as being in opposition 
to the search for and production of additional domestic energy 
supplies. . . .As a freshman in a competitive political situation 
and as a newcomer to the complexities of the energy industry, 
he is a prime candidate for factual information. ^ 
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Perhaps the most bizarre example of the use of roll calls in the 1976 



election campaign was the ratings compiled by the National Alliance of Senior 
pitizens (NASC) and the National Council of Senior Citizens (NCSC). The NASC 

I 

f Golden Age Index” rates Congressmen based on votes "perceived most demon- 
strative of a commitment to fiscal responsibility. Republicans scored exceed- 
jingly high in this index. The NCSC index, on the other hand, is concerned more 
jwith governmental programs in support of the elderly, such as pension reform 
jand Medicare benefits. Not surprisingly, high scores on this index rarely 
coincide with high scores on the NASC index. In one heated California debate 

I 

jbetween two candidates, a liberal incumbent was accused of "not caring about 

the elderly" based on his NASC (fiscal responsibility) score. Although he was 

quick to point out his high score on the NCSC index and secure an apology from. 

his opponent, the incident points out not only the increased use of roil call 

analysis in politics but also the problems which occur when poorly constructed, 

invalid and .misleading indices are essentially misused. 

What is required is some sort of quality control of the "rating game. " 

A good example of suggested control is a May 1976 study entitled The Rating Game , 

published by the House Republican Research Committee. This report recom.mends 

a "code of ethics , " several parts of which are relevant to the problems of roll 

call analysis addressed in this article. Their code includes 

an objective, rational and understandable vote selection 
process, a guarantee of membership input in the determina- 
tion of the rating group's concerns, detailed vote descrip- 
tions , avoidance of percentage compilations , and tech- 
nical accuracy in description and tabulations . ^ 
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Political scientists may legitimately ask to what extent the scholarly work on 
roll call analysis has addressed and perhaps alleviated the problems cited. 

It is to such an inquiry that we now turn, examining briefly what has been 
done and its applicability to the policymaking arena. 

Roll Call Analysis in Political Science 

g 

No attempt will be made to duplicate the several textbooks and articles 
which adequately survey the field of roll call analysis. However, a brief sum- 
mary of some of the more commonly used methods will shed some light on the 
general problem of using this type of analysis for policymaking. 

By far the most frequently used method of using roll calls to describe 
and explain legislative behavior is the simple percentage technique. Specifi- 
cally, the various Congressional Quarterly indices are the data used in many of 
the major studies of legislative behavior.^ This method assumes that each roll 
call is equally weighted and is an attractive technique since the simple percen- 
tage is a ratio measure which facilitates comparison across legislators and ti.me, 
and can be used in a variety of statistical techniques . 

p 

Another popular method is Guttman scaling, especially since the advent 
of statistical computer packages which facilitate construction of the scales . It 
has an advantage over simple percentage scales in that it empirically checks for 
unidimensionality of the roll calls selected. A drawback of this method is that 
it produces only an ordinal scale. This has not stopped many a political scien- 
tist from assuming interval data and proceeding to invalidly use such scales in 

q 

regression analysis . ^ 
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One of the disadvantages of the Guttman scaling approach is the lim- 
ited number of roll calls which can be used to construct a scale (SPSS limits 
the analyst to 12). Factor analysis^^is a third method of constructing scales 
i/yhich alleviates this problem. As with the two previous methods, each roll 
sail is equally weighted. The ability to process a greater number of roll calls 
makes the selection and grouping of roll calls by issue-area somewhat more 
objective than the Guttman technique, where some sort of subjective sorting 
(process must be used in order to get the number of roll calls to a workable size. 

It should be added here that a significant amount of academic work has been 
done comparing the statistical properties and structures of Guttman scale and 
■factor solutions Warwick's evaluation of the two techniques concludes that 
factor analysis is 

a refinement over Guttman scaling in the assessment of roll 
call similarity, although where this e>dra precision is not 
required, Guttman scaling remains an acceptable methodology. 

There have been several other methods of roll call analysis which have 

been developed . Specifically, multi-dimensional scaling, cluster-bloc 
14 15 

analysis and smallest space analysis. Without judging their future 
utility in describing or explaining legislative behavior, it can be said that 
they have not been used to any great extent to this point. In addition, the three 
methods previously summarized will serve to make the next point in this article, 
namely that despite their statistical rigor, these methods fall short as techniques 
which can be utilized in policymaking . 
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What are the shortcomings of these methods? First, and most crucial 



to politics, is the selection of issues to make up the indices and scales. In 

1976, there were 688 roll call votes in the Senate. If the policy problem is to 

determine where a senator stands in regard to defense policy, which votes are 

selected? To use Guttman scaling, a small subset of votes must be selected. 

As Warwick points out, Guttman scaling is extremely sensitive to the vote 

16 

splits in each roll call. Out of 30 defense roll calls, 10 may scale, 10 may 
not, simply based on the Yes-No breakdown on the vote. Factor analysis 
improves the selection process somewhat, in that it can handle a larger number 
of roll calls (although not the 688 roll calls in 1976). In my dissertation research 
I found that factor analysis is more discriminating than Guttman scaling (e.g. 
some military assistance roll calls appeared in the Guttman scales but formed 
a distinct cluster when factored.) However, the problem which still remains 
is insuring that, on any given roll call, a "Yes" is pro — or anti-defense. Was 
the vote a procedural protest of some sort? Clearly, both Guttman and factor 
analysis cannot address this problem. 

A second problem with the three commonly used methods of roll-call 
analysis is the assumption that each roll call adds equally to the dimension 
being measured. Some votes are seminal, others on the same issue are trivial. 
While these techniques can, over a large set of roll calls, give the general 
orientation of a legislator, this is not adequate for policy purposes. This is 
particularly noticeable in the defense issue-area. Senators Cranston and Tunney 
would come out In any general evaluation as somew'hat critical of the Defense 
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Department. Yet on the several roll calls to cut the B-1 bomber, they were very 
isupportive of continuing the program. 

I 

I 

There has been one recent attempt to solve this problem of determining 

the relative importance of roll calls. Using the House of Representatives as 

ibis source of data, Frans Bax has proposed an Index of Importance which 

, relies upon the practice of the House to allocate varying 

amounts of time for debate on legislation coming to the 
, floor. . . .Bills receiving a rule are given a value on the 

Index equal to the number of hours granted for a debate. 

|Bax then takes his index and finds that it correlates it with those attributes of 

legislation that suggest importance — occurrence of a roll call vote on the bill, 

frequency of roll call votes on the bill, partisan controversy, controversy within 

committee, number of legislators voting. Presidential involvement, and floor 

action in the form of amendments to the bill.^^ Bax concludes that his "Inde.x is 

closely and explicitly linked to the judgments of those House members who are 

20 

expected to assess the relative importance of many bills . " 

Bax has overcome some of the obstacles to using roll calls in policy 
analysis, particularly the problem of constructing unobtrusive measures of 
importance. But his goal is similar to other approaches in that he is attempting 
to create a picture of how the House "really" feels about the relative importance 
of roll calls. His Index of Importance is an important step forward in under- 
standing the House, but does not address the concern of this article. Namely, 
how can we evaluate the interaction of the Congress with outside groups (interest 
groups, executive agencies, etc.) using roll call analysis? What the House 
deems important in defense policy (e.g. lengthy debate on the neutron bomb in 
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the summer of 1977) may not be as important to the Defense Department, or 
specific groups within DoD such as the Navy. 

The missing ingredient in all the approaches is the expert judgment which 
must go into both the selection and weighting process. In terms of selection, 
most organizations constructing voting indices do have a panel which selects 
votes but rarely do they specify why the votes were selected, and why they 
are a good indicator of the dimension being measured. This can only lead to 
increased suspicion that votes are being selected to create a pattern for political 
gain, a suspicion which leads to little use of the index. Although somewhat 
general for most policy purposes, the ADA index has been used consistently 
for many years as a credible index of a liberal voting record. As far as weighting 
is concerned, little or none is conducted. I think the main reason for this is 
that in their quest for objectivity, the raters feel that distinguishing the impor- 
tance of roll calls is blatantly subjective, and hence don't even attempt it. The 
major problem is that the search for objectivity is a futile one. A legislator 
does not possess one "objective" attitude toward environmental, defense or 
farm issues. Rather, there are various versions of this attitude which vary 
according to the perceptions of the conflicting groups which determine policy in 
accordance with what Allison has termed bureaucratic politics . If this is how 
policy is made, and a legislator's voting record is to be used as a measure of 
previous stances on issue-areas, it is obvious that attention must be paid to how 
each group views the voting record. This means systematically tapping expert 
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opinion regarding which votes make up an issue-area, and their relative weight 
in contributing to the dimension being measured. It is to this task that we now 
turn. 

The Use of Judgment Data In Roll Call Analysis 

Having concluded that expert opinion must be systematically measured 
if roll call analysis is to be used in the political process, we now address spec- 
ifically how this can be accomplished. The two basic steps involve the selection 
of votes for a particular issue-area or dimension, and the weighting of those 
votes in terms of their importance to the group making the index or scale. This 
.article is more concerned with the second step, so the first step of vote selection 

i 

will be only briefly mentioned. Obviously some sort of Delphi technique could 
be used to arrive at ar consensus as to what votes make up the dimension in 
question. If such a selection is destined to be the rating organization's view, 
care must be taken to insure that the judges selected to evaluate roll calls in 
some way represent the organization and produce a true "party line. " How this 
is done is much more a matter of organizational behavior and beyond the scope of 
this article. 

But once agreement is reached on what roll calls make up a certain 
dimension, some sort of weighting procedure must be used so that a valid picture 
of a legislator's record is constructed. Two such procedures were used in this 
study . 
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The Method of Paired Comparisons 



The basic idea in this method is that a judge can rank any two roll calls 

as to their relative importance. If the scale being constructed contains n roll 

calls, each judge will make n(n-l)/2 comparisons. The paired comparison 

method converts these ordinal judgments into an interval scale. The method is 

not new, having been in the methodological kits of psychologists since Thurstone 

? 1 

spelled out the Law of Comparative Judgement in 1927. There have been many 

O O 

applications of this method in political science but none which address roll 
call voting. An example from the several experiments conducted during this 
research will illustrate the method. 

Seven Senate roll calls on defense policy were selected by the rating 
organization (in this case, I selected them for experimental purposes) as indica- 
tors of general attitude toward the Defense Department. The judges were given 
a brief description and background on the vote, along with the recorded results 
of the vote. They were instructed to assume the role of Secretary of Defense in 
1976, and then rank the votes in terms of their importance to you as Secretary of 
Defense. No ties were permitted. Two groups of judges were used. One was 
my class in data analysis which had taken two courses, one in international 
relations and another in American national security policy within the last six 
months (hereafter referred to as Group NAVAL) . The other group was made up of 
participants in the National Security Education Seminar held each summer at 
Colorado College, a group with considerably more expertise and knowledge in 
national security affairs (referred to as Group NSES). 
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They were given a sheet of paper in the following format: 
Most Important 



Least Important 

This raw data was then transformed into a matrix as shown in Table I. 
The data can be interpreted as follows. There were 4 out of 6 judges who felt 
vote #12 was more important than #6, while only 1 judge thought #9 was more 
Important than #6 . 

These raw frequency rankings are next transformed into an array of pro- 
portions , in which each cell entry (Pj^) is equal to the proportion of times roll 
call k was judged more important than roll call j. (See Table 2) 

The next step involves transforming these proportions into Z scores 
using the table of normal distribution. The cell entries (Zj}.) now equate to the 
unit normal deviate corresponding to the proportions in Table 2. (See Table 3) 
The final step in the process is the computation of a scale score for 
each roll call, and is also included in Table 3 above. As can be seen, the 
scale score for each roll call is the mean of each column. While the reader is 
referred to Torgerson^^ for the theory and assumptions implicit in the technique, 
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Table 1 



(j) 



Raw Data Matrix For Paired Comparison Technique 

Roll Call # (k) 





6 


7 


9 


12 


13 


15 


16 


6 


- 


1 


1 


4 


1 


2 


1 


7 


5 


- 


3 


4 


1 


3 


4 


9 


5 


3 


- 


4 


1 


4 


2 


12 


2 


2 


2 


- 


2 


2 


2 


13 


5 


5 


5 


4 


- 


4 


4 


15 


4 


3 


2 


4 


2 


- 


3 


16 


5 


2 


4 


4 


2 


3 


- 
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Table 2 



Roll 

Call 

(j) 



Proportion Matrix For Paired Comparison Technique 

Roll Call (k) 





6 


7 


9 


12 


13 


15 


16 


6 


- 


.20 


.2 


.667 


.2 


.333 


.2 


7 


.833 


- 


.5 


.667 


.2 


.5 


.667 


9 


.833 


.5 


- 


.667 


.2 


.667 


.333 


12 


.333 


.333 


.333 


- 


.333 


.333 


.333 


1'3 


. 833 


.833 


.833 


.667 


- 


.667 


.667 


15 


.667 


.5 


.333 


.667 


.333 


- 


.5 


16 


.833 


.333 


.667 


.667 


.333 


.5 
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Table 3 



Z Score Matrix For Paired Comparison Technique 

Roll Call (k) 







6 


7 


9 


12 


13 


15 


16 




6 


- 


-.841 


CO 

! 


.43 


-.841 


-.43 


-.841 




7 


.965 


- 


0 


.43 


00 

f 


0 


.43 




9 


.965 


0 


- 


.43 


1 

00 


.43 


-.43 


Roll 

Call 

(j) 


12 


CO 

1 


-.43 


CO 


- 


-.43 


-.43 


-.43 




13 


.965 


.965 


.965 


.43 


- 


.43 


.43 


• 


15 


.43 


0 


CO 

• 

1 


.43 


-.43 


- 


0 




16 


.965 


-.43 


.43 


.43 


-.43 


0 


- 


n 

V" 




















2jk 


.5514 


-.1051 


-.0437 


.3685 


-.5447' 


0 


-.1201 



J 
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we can and should look at what has been done here. The best way to do this 
:is look at the highest and lowest scores. The roll call with the highest score 
is #6. It received that score due to the basic fact that in most cases the judges 
ranked it above all others. Roll call i^l3 received the lowest score because in 
most cases it ranked below the other roll calls . Roll call #15 is in the middle of 
the scale since overall the judges were ambivalent concerning its importance. 

The resulting interval scale can be transformed to make it more inter- 
pretable. In this case, .5447 was added to each scale score resulting in the 
following scale of importance for the 7 defense policy roll calls . (See Table 4) 

Since these now are interval data, some basic conclusions can be 
jmake relative to the importance of the roll calls . Mainly, there appears to be 
three levels of importance — 6 and 12, 15, 9, 7, and 16, and 13. The problem is 
that very little else can be done with the scale. The purpose behind weighting 
the votes was to apply them to individual legislators so that a "defense" score 
could be produced. Since these are interval and not ratio data (i.e. , the 0 
point is not meaningful since it does not indicate an absence of the dimension), 
one cannot modify (multiply) a 'Yes ' vote by the weight of the roll call. An 
illustration makes the point. Listed in Table 5 are the results of our experiment 
so far. Scale A is the original scale, scale B has added +1 to each value. The 
intervals in both scales are identical. Now we apply both scales to three 
hypothetical voting records . 
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Table 4 



Defense Support Scale, Paired Comparison Technique 



Roll Call 
# 


Subject of Roll Call 


Scale 

Score 


6 


Cut off funds for B-1 


1.0961 


12 


Conference Report - Defense Procurement 


.9132 


15 


Cut funds for Lance missile 


.5447 


9 


Cut off funds for Minuteman - Auth 


.501 


7 


Delay B-1 decision until Feb 77 


.4396 


16 


Cut off funds for nuclear carrier 


.4246 


13 


Defer decision on Minuteman 


.00 



Table 5 



Adjusted Defense Support Scale, Paired Comparison Technique 



Roll Call 


Scale A 


Scale B 


6 


1.0961 


2.0961 


12 


.9132 


1.9132 


15 


.5447 


1.5447 


9 


.5010 


1.5010 


7 


.4396 


1.4396 


16 


.4246 


1.4246 


13 


.00 


1.00 
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(+) = pro-defense vote 



^ 15. ^ Z !§. Score A Score B 

Cong. X++-++-+ 2.9499 7.9499 

Cong. Y+ + — — 2.0093 5.0093 

Cong. Z---- + -- .4396 1.4396 

To see what has happened here, we compare the intervals between 
Congressmen X and Y, and Y and Z, for both scales A and B. They should be 
proportional. But they are not. 

(2.9499-2.0093) ^ (7.9499-5.0093) . .9406 ^ 2,9406 . 5992 ?^ 8237 

(2.0093-.4396) (5.0093-1.4396)' 1.5697 ^ 3.5697' 

While the paired comparison method can shed some light on the relative 
importance of roll calls to a group of judges , the resulting interval scale cannot 
be used for the ultimate purpose of rating legislators on a specific dimension. 

The Constant Sum Method 

The key to the solution of the above problem is to obtain ratio data from 
those judging the roll calls. Several alternatives exist. The first is to tell the 
judges to give each roll call a value from 0 to 1.0 such that the final results 
reflect the ratios among roll calls. If one roll call is .4 and another .8, the 

24 

latter contains twice as much of the property (e.g. anti-defense) as the former. 
Experience with this technique has shown that it is very difficult for judges to 
think in ratio terms, especially with more than a few cases to evaluate. This 
has led to the so-called "pie allocation" approach, in which judges are given 
a pie of 100 points , and asked to split it up so that each piece not only has a 
size relative to all others, but the graphic aspect of the technique gives more 
assurance that the judges are thinking in ratio terms. 
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There exists a third alternative, however, which also produces ratio 

data but puts much less of a demand on the judge. This method is generally- 

termed direct estimation of sense ratios, or more specifically, the constant sum 

2 7 

method. As with the paired comparison technique, it is not a new method. 

The stimuli (in our case the descriptions of the roll calls) are presented to the 
judges in pairs . The judge is instructed to divide 100 points between them in 
accordance with the absolute ratio of the greater to the lesser. Although the 
judge still must think in ratio terms (assigning 80 points to one roll call and 20 
to the other indicates the former is 4 times as "important" as the latter), his task 
is simplified since he only must deal with one ratio at a time. Since the judge 
does have the opportunity to assign 0 points to a roll call, a natural 0 point in 
fact exists . 

How these raw data are converted into scale scores can best be describee 
by returning to the previous set of roll calls and judges. Having simply ranked 
the roll calls in importance, the judges were then given the roll calls in pairs 
and asked to split 100 points between them. They were told that an 80-20 split 
between a B-1 issue and a nuclear carrier issue would indicate that the B-1 roll ^ 
call is 4 times as critical to DoD as the nuclear carrier roll call in determining 
whether a senator is "pro" or "anti" defense. ' 

The first matrix produced (Table 6) is a matrix V in which each cell (Vjj,) 
is the average number of points assigned to r'oll call k when compared to roll call 

j. 
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Table 6 



Matrix V With Elements Denoting The Average Number Of 
Points Assigned To Roll Call k When Compared To Roll Call j 

Roll Call k 





6 


7 


9 


12 


13 


15 


16 


6 


50.0 


24.3 


27.9 


55.7 


19.3 


32.2 


50.0 


7 


75.7 


50.0 


52.9 


74.7 


41.2 


51.4 


50.3 


9 


72.1 


47.1 


50.0 


75.0 


22.9 


50.0 


50.8 


12 


44.3 


25.3 


25.0 


50.0 


19.3 


27.1 


29.3 


13 


80.7 


58.9 


67.1 


80.7 


50.0 


59.3 


67.1 


15 


67.9 


48.6 


50.0 


72.9 


40.7 


50.0 


40.0 


16 


50.0 


49.7 


49.3 


70.7 


32.9 


60.0 


50.0 
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The next step is to construct a matrix W in which each cell is the ratio 



of the average points indicated by the column to those indicated by the row (Table 
7) . (Wjk= 

Vkj 

The final step involves calculating the geometric means of the columns 

O p 

of the above matrix. ° The scale values for each roll call (Sj) are calculated 

using the following equation: 

n 

log Sk =21 log Wj-k 

JZI 



n 

where n = the number of roll calls . 

The actual scale values are the antilogs of the above values , calculated 
as follows: 



Sk = 



^^jk 



1/n 



The scale value for roll call 6 is calculated as follows: 



Sg = [(1)(3.118)(2.589(.794)(4.184)(2.110)(1)]^/’^ = 1.78 
Us'ing the above approach, tv/o sets of scales were constructed; one 
using the experts from the National Security Education Seminar (NSES) and the 
other using naval officers from my data analysis class (NAVAL). The following 
table gives the results of these two experiments. (See Table 8) 

In a quick test of the rank orders of the four scales, a non-parametric 
test (Kendall's tau) was conducted to determine the level of association of the 
four scales . (See Table 9) 
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Table 7 



Roll 

Call 

J 



Matrix W With Elements Denoting The Ratio Of Average Points 
Indicated By The Column To Those Indicated By The Row 

Roll Call k 





6 


7 


9 


12 


13 


15 


16 


6 


1 


.321 


.386 


1.259 


.239 


.474 


1.00 


7 


3.118 


1 


1.12 


2.958 


.699 


1. 059 


1.012 


9 


2.589 


. 892 


1 


.667 


,340 


1.00 


1.028 


12 


.794 


.338 


.333 


1 


.239 


.373 


.414 


13 


4.184 


1.430 


2.937 


4.181 


1 


1.457 


2.044 


15 


2.110 


.944 


1.00 


2.683 


.686 


1 


.667 


16 


1.00 


.988 


.972 


2.414 


.489 


1.5 


1 



- 21 - 



Table 8 - Defense Support Scales, Constant Sum Method 



NSES NAVAL 

Roll Call (n= 6 ) (n=19) 

Si 2 - Conference Report - Defense Procurement 1.826 1.664 

Se - Cut off Funds for B-1 1.780 1.683 

Sj 0 - Cut off Funds for Nuclear Carrier .927 1.098 

Sg - Cut off Funds for Minuteman - Auth .881 1.082 

Si 5 - Cut off Funds for Lance Missile .880 .688 

Sy - Dalay B-1 Decision Unitl February 77 .746 .736 

8^3 - Defer Decision on Minuteman .463 .594 



Table 9 

Rank Order Correlation of Guttman, 

Paired Comparison And Constant Sum Scales 

Paired 

Guttman NSES NAVAL Comparisoi 



Guttman 


1 


.428 


.238 


.523 


NSES 




1 


.857 


.571 


NAVAL 






1 


.523 



Paired 

Comparison 



1 



The two constant sum scales were highly related, whereas the other 



•elationships were much lower. We have now produced ratio data which not only 
iifferentiates each roll call in terms of its importance, but also in a form that 
ban be used in assigning scores (in this case "anti-defense" scores) to indi- 
./idual legislators. Some examples using 1976 votes will demonstrate the use of 
;ihe various techniques . (See Tables 10 and 11) 

In each of the tables, the "Guttman" score is a simple index of the 
number of anti-defense votes (common practice in most Guttman scale analyses) . 

In Table 10, the scale scores for the "pure" types of senators are listed. The 
fifirst conclusion drawn is that using tha rank-order scores produced by the Guttman 
procedure as interval data would be seriously misleading. Second, there are some 
significant differences when Senator Burdick is rated by the two groups of 
experts (NSES vs NAVAL), If NAVAL represents either an interest group or gov- 
ernmental bureaucracy which keys their activity to anti-defense activity, they 
will perhaps be watching Burdick more closely than NSES. 

* The key conclusions, however, are drawn from Table 11, in which the 
scores of "mixed" types are listed. It should simply, but significantly, be noted 
that senators with identical Guttman scores have different NSES and NAVAL scores, 
and conversely, similar NSES and NAVAL scores produce dissimilar Guttman scores. 
In the case of mythical Senator C the lower Guttman score produces a higher 
NSES and NAVAL score. Clearly, the unidimensionality test implicit in Guttman 
scaling is inadequate in accurately depicting the magnitude of the "anti-defense" 
dimension for individual senators. 
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Table 10 



Constant Sum Scale Scores For Pure Type Senators 
(+) - Anti-Defense Position on Roll Call 

Roll Call Number 



Guttman NSES NAVAL 



Senator 


12 


15 


6 


9 


13 


16 


7 


Score 


Score 


Score 


Buckley 


- 


- 


- 


- 


- 


- 


- 


0 


0 


0 


Nunn 


- 


- 


- 


- 


- 


- 




1 


.746 


.736 


Magnuson 


- 


- 


- 


- 


- 






2 


1.673 


1.834 


Burdick 


- 


- 


- 








-4- 


4 


3.017 


3.510 


Hartke 


- 


- 


+ 


+ 


+ 


-»■ 


-4- 


5 


4.797 


5.193 


Leany 


- 


+ 


+ 


•f 


4- 


+ 




6 


5.677 


5.881 


McGovern 


+ 




+• 


+ 




+ 




7 


7.503 


7.545 



Table 11 



Constant Sum Scale Scores For Mixed Type Senators 
Roll Call Number 



Senator 


12 


15 


6 


9 


13 


16 


7 


Guttman 

Score 


NSES 

Score 


NAVAL 

Score 


Goldwater 


- 


- 


- 


- 


- 


4- 


- 


1 


.327 


1.098 


Wclcker 


- 


- 


- 


4- 


4* 




- 


2 


1.344 


1.676 


Percy 


- 


- 


- 


+ 


4- 


- 


4* 


3 


2.090 


2.412 


Randolph 


- 


- 


- 


-4- 


- 


4- 


4- 


3 


2.554 


2.755 


Schwelker 


- 


- 


-4- 


4* 


4- 


- 


4- 


4 


3.870 


3.964 


Cranston 


- 


-4* 


- 


4- 


4- 


> 


- 


4 


3.151 


3.462 


Hatfield 


-4- 


- 


-4- 


4* 


4- 


4- 


4- 


6 


6.623 


6.857 


A 


- 




>4* 


4- 


- 


4- 


- 


4 


4.416 


4.551 


B 


- 




- 


4- 


4- 


- 


4* 


4 


2.370 


3.100 


C 


-4- 


*4* 


- 


- 


- 


- 


- 


2 


3,606 


3.347 


D 


- 


- 


- 


4- 


4* 


4* 


4- 


4 


3.017 


3.510 
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Conclusions 



The methods put forth in this article are intended to spur discussion and 
esearch regarding how roll call data can and should be employed in the policy- 
laking arena. It has been assumed that the bureaucratic politics model accurately 
vixplains policy outputs, that policy is a function of bargaining, power and influence, 
.legislators' votes are a record of their position, and as such form the "previous 
stances" useful in the bargaining which constitutes politics. This is not to say 
:hat there are not serious normative questions concerning the permanent adoption 
;3f this model, or that the search for an objective measure using roll calls is 
lot a desirable goal. Rather, what has been said here is that bureaucratic politics 
is descriptive of how policy is made, roil call analysis (however crude) is part 
of the currency in such policymaking, and more attention should be paid to 
designing methods which take into account the selection and weighting of roll 
calls by experts representative of the organizations involved. 
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